On the Suitability of Numerical Performance Measures for Class Imbalance Problems

The class imbalance problem has been reported as an important challenge in various fields such as Pattern Recognition, Data Mining and Machine Learning. A less explored research area is related to how to evaluate classifiers on imbalanced data sets. This work analyzes the behaviour of performance measures widely used on imbalanced problems, as well as other metrics recently proposed in the literature. We perform two theoretical analysis based on Pearson correlation and operations for a 2× 2 confusion matrix with the aim to show the strengths and weaknesses of those performance metrics in the presence of skewed distributions.